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(54) INFORMATION PROCESSING APPARATUS AND METHOD, 
MEDIUM 

PROBLEMS BE SOLVED: To dispense with jump required 
for switching the reading main and sub streams, to eas.ly 
perform synchronized reproduction of the main stream and 
the sub stream and to easily separate each elementary 

SOLUTION: A flowchart denoting a processing flow of a 
recording method includes: a step S101 of dec.dmg ; PID 
values of all elementary streams of a mam transport stream 
and a sub transport stream to be values not overlapped I w.th 
each other; a step S102 of referring to a common STC 5 to 
encode the main and sub transport streams, where.n the PIDs 
of PCR packets are made identical in the ma.n stream and the 
fub stream and contents of program map tab.es ,n the ma.n 
stream and the sub stream are made identical; and a step 
sl03 of interleaving the main stream and the sub stream and 
recording the interleaved streams to a recording medium. 
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A . Relevance of the AWe-identified Document 

This document discloses prior art as technical 
background of the present invention. 

B . Translation of the Relevant Passages of the Document, 

See also the attached English Abstract. 

The following is a partial English translation of 
exemplary portions of non-English language information 
that may be relevant to the issue of patentability of the 
claims of the present application. 

[0023] 

[DESCRIPTION GF THE EMBODIMENTS] 
The following explain an embodiment of the present 
invention with reference to figures. Fig. 3 illustrates an 
example of recording data generating method of the 
present4nvention,-The methodJs . teM^M^i^J^L. 
a main multiplexing audio video stream file (hereinafter, 
referred to as "main AV stream") such as audio video; and 
(ii).a sub multiplexing stream file (hereinafter, referred to 
as «sub stream") accompanied with the main multiplexing 
audio video stream file. Examples of the sub multiplexing 
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stream file includes subtitles, graphics and the like. Such 
streams are in compliance with, e.g., the MPEG2 program 
stream format. The streams are interleaved and are 
recorded onto a storage medium as shown in Fig. 3. 



In Fig. 3, the main AV stream is divided into data 
units la, lb, lc, ... each having a predetermined time 
corresponding to, e.g., 1 GOP (Group Of Picture). Likewise, 
the sub stream is divided into data units 2a, 2b, 2c, ... 
each having a predetermined time corresponding to 1 GOP 
or the like. This makes it possible that, e.g., the data unit 
la of the main AV stream and the data unit 2a of the sub 
stream are reproduced in synchronism. Further, the main 
AV stream and the sub stream are interleaved such that 
selection of the data units of the main AV stream and 
selection of the data units of sub stream are alternately 
carried out. This allows formation of one stream, and the 
stream is recorded onto the disc. Note that the recording 
may be carried out such that each data unit of the sub 
stream comes before each data unit of the main stream. 
Further, the main stream and the sub stream are encoded 
in reference to a common time reference signal such as 
system time clock STC (system time clock; a reference 
synchronization signal). This makes it easier for the main 
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stream and the sub stream to be reproduced in 
synchronism. 

[0053] 

Fig. 9 is a flowchart illustrating a flow of 
reproduction processing in the player system shown in Fig. 
8. In a step S201, the two transport streams, interleaved 
and stored in the medium, are read out in order of the 
data allocation. 

[0054] 

In a step S202, the readout data is sent to the 
de-multiplexer 53. 
[0055] 

In a step S203, the de-multiplexer 53 removes an 
elementary stream from the received data, and sends the 
removed elementary stream to the decoder. Specifically, 
the de-multiplexer 53 removes the audio stream and the 
video stream, and sends them to the AV decoder 55. 
Further, the de-multiplexer 53 removes the graphics 
stream and the subtitle stream from the sub stream, and 
sends them to the graphics-subtitle decoder 56. 

[0056] 

In a step S204, each decoder carries out decoding in 
disregard of the DTS. In other words, the AV decoder 55 
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decodes the audio data and the video data, and the 
graphics-subtitle decoder 56 decodes the graphics data 
and the subtitle data. 
[0057] 

In a step S205, the audio data and the video data 
thus decoded are sent to the video 
composer/sync-controller 57 such that the video data and 
the graphics data are combined. This causes generation of 
a display video signal. Further, the audio video is 
outputted in synchronism in reference to the PTS. 

[0058] 

The following explains an example of random access 
reproduction when the main stream and the sub stream 
are interleaved and stored in the disc medium. The 
random access reproduction becomes easy by interleaving 
and recording (i) main stream data corresponding to a 
predetermined time, and (ii) the sub stream which is to be 
reproduced in synchronism with the main stream data. 

[0059] 

Fig. 10 explains a random access reproduction 
operation. A reference symbol AD1 indicates an address of 
a packet from which a subtitle and graphics start to be 
reproduced in synchronism with an I picture of target 
reproduction time (PTS) in the random access 
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Data readout is started from the address AD 1 of the 
packet from which the sub stream starts to be reproduced 
in synchronism with the I picture of target reproduction 
time (PTS) of the random access reproduction. Because 
the main stream and the sub stream are consecutively 
stored, no jumping operation is required for the switching 
between the readout of the main stream and the readout 
of the sub stream. This allows reproduction of the target 
video, and of the sub stream to be reproduced in 
synchronism with the target video. 
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